A multi-row deletion diagnostic for influential observations in small-sample regressions

Policy makers often look to economic research for guidance. Econometric studies examining actual economic outcomes for countries under different historical policy regimes are particularly valuable. These studies are sometimes performed on small (N ≈ 100) data sets. A particular concern when working with these data sets is the potential for the inference from the regression to be sensitive to a few influential data points. For context, consider the case of growth regressions. Growth regressions have been criticized on many levels, one of which is their instability as the sample is altered in small ways. In many cases, small changes in sample lead to a reversal of the economic inference from the modeling exercise. In one example, a well-cited study finding large benefits of investment on economic growth was shown to hinge on the inclusion of Botswana in the sample. Another study’s finding that high public debt to GDP ratios cause measurably slower economic growth, an outcome that heavily influenced political platforms in elections in the United States and Great Britain, was later shown to hold only if New Zealand is included in the sample. Despite this, the sensitivity of economic inference to sample has received little systematic attention and is rarely rigorously explored by growth researchers. The authors’ premise is that researchers and policy makers are interested in knowing when the policy inference from a regression analysis is driven by a single observation or small group of observations, regardless of what the formal regression statistics say for their baseline regression. For example, for small changes in sample, do the signs on the coefficients of interest change? Does the magnitude of a coefficient move from being meaningful to irrelevant? Does the t-statistic change from signaling statistical significance to signaling statistical insignificance? In this paper the authors present a multi-row deletion analysis (MRDA) data analytics approach to systematically test small-sample regression results for the presence of influential points or groups of influential points. The approach is complementary to two other approaches often used in such diagnostics: DFBETAS and robust regression. Using both simulated and real data, the authors show that MRDA provides insights into sample sensitivity that these other approaches miss. In their real data analysis, they take a second look at the data used in a well-cited paper examining the impact of institutional quality on the resource curse. That paper suggested that resource-rich countries with good enough institutional quality had accelerated growth as a result of their resource endowment. Countries with poor institutional quality had slower growth. The World Bank makes frequent note of this result when it discusses developing institutional capacity in resourcerich developing nations.